which NoSQL for billions of records [closed]

Posted by airtruk on Server Fault See other posts from Server Fault or by airtruk
Published on 2012-11-01T03:24:22Z Indexed on 2012/11/01 5:04 UTC
Read the original article Hit count: 437

Filed under:

mongodb

|

cassandra

|

nosql

There are plenty of discussions around NoSQL databases around and a lot of them are about data logging in the social media section. The problem I'm trying to solve falls more into the scientific computing section, where I have several 1000s of billions of pieces of information that I want to query with different a different criteria for each query.

All data is at least a 4 dimensional space, which means I have a 3D location (x,y,z) and a time component - plus the value and unit. Say temperature at xyz and 10min in degree Celcius. A typical query result may contain several million results ...

I have read about pretty much all NoSQL solutions being exceptionally fast for inserting records, but when it comes to querying them it's a different story. I'm leaning towards MongoDB for the implementation and platform for developing the necessary code since it is more closely related to the current solution using MySQL. Happy to be proven wrong though when it comes to the choice of the NoSQL solution.

© Server Fault or respective owner

Related posts about mongodb

MongoDB usage best practices

as seen on ASP.net Weblogs - Search for 'ASP.net Weblogs'
The project I'm working on uses MongoDB for some stuff so I'm creating some documents to help developers speedup the learning curve and also avoid mistakes and help them write clean & reliable code. This is my first version of it, so I'm pretty sure I will be adding more stuff to it, so stay tuned… >>> More
Errors trying to run MongoDB

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I'm running Ubuntu Server 12.04 (32 bit) on an old (1998) computer. Everything's working fine until I try and start MongoDB. somekittens@DLserver01:~$ mongo MongoDB shell version: 2.2.2 connecting to: test Sun Dec 16 22:47:50 Error: couldn't connect to server 127.0.0.1:27017 src/mongo/shell/mongo… >>> More
push new value to mongodb inner array - mongodb/php

as seen on Stack Overflow - Search for 'Stack Overflow'
hi i have this document in mongo: { "_id": ObjectId("4d0b9c7a8b012fe287547157"), "done_by": ["1"] } and i want to add another value to "done_by" field, so my expected document will be:: { "_id": ObjectId("4d0b9c7a8b012fe287547157"), "done_by": ["1","2","3"] } i try this: $conn… >>> More
Write-only collections in MongoDB

as seen on Stack Overflow - Search for 'Stack Overflow'
I'm currently using MongoDB to record application logs, and while I'm quite happy with both the performance and with being able to dump arbitrary structured data into log records, I'm troubled by the mutability of log records once stored. In a traditional database, I would structure the grants for… >>> More
How to install mongoDB on windows?

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi! I am trying to test out mongoDB and see if it is anything for me. I downloaded the 32bit windows version, but have no idea on how to continue from now on. I normally use the WAMP services for developing on my local computer. Can i run mongoDB on Wamp? However, what's the best (easiest!) way… >>> More

Related posts about cassandra

Cassandra inserts using Net::Cassandra::Easy in Perl

as seen on Stack Overflow - Search for 'Stack Overflow'
When using the Perl module Net::Cassandra::Easy to interface with Cassandra I use the following code to read colums col[123] from rows row[123] in column-family Standard1: my $cassandra = Net::Cassandra::Easy->new(keyspace => 'Keyspace1', server => 'localhost'); $cassandra->connect(); my… >>> More
How do I insert a row with a TimeUUIDType column in Cassandra?

as seen on Stack Overflow - Search for 'Stack Overflow'
In Cassandra, I have the following Column Family: <ColumnFamily CompareWith="TimeUUIDType" Name="Posts"/> I'm trying to insert a record into it as follows using a C++ generated function generated by Thrift: ColumnPath new_col; new_col.__isset.column = true; /* this is required! */ new_col… >>> More
shell script over SSH ends unexpectedly after running 'ant build'

as seen on Super User - Search for 'Super User'
I wrote a shell script that runs on remote host to build source code with 'ant build' command, and then distribute the built binary to other servers. However, right after Ant build is over successfully(I can see the command line output saying Build was successful), the ssh session ends and whatever… >>> More
Cassandra Remote Connection

as seen on Server Fault - Search for 'Server Fault'
I'm not managing to connect to cassandra from outside machines. The database is hosted on a windows machine and im trying to connect through a mac (but this shouldn't cause problems) Local connection works: C:\cassandra\bin>cassandra-cli Starting Cassandra Client Connected to: "Test Cluster"… >>> More
A catalogue of Cassandra log messages: What is the correct interpretation?

as seen on Stack Overflow - Search for 'Stack Overflow'
The following is a complete catalogue of all log messages generated by Cassandra 0.6 when stress-testing a Cassandra installation over an extended period of time: AntiEntropyService: Sending AEService tree for (,) to: [] CassandraDaemon: Binding thrift service to localhost/N.N.N.N:N CassandraDaemon:… >>> More